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Abstract — The Uncertainty Representation and Reasoning 
Evaluation Framework (URREF) includes an ontology that 
represents concepts and criteria needed to evaluate the uncertainty 
management aspects of a fusion system. The URREF ontology defines 
self-confidence as a measure of the information credibility as 
evaluated by the sensor itself The concept of confidence, which is not 
explicitly defined in the ontology at URREF, has been extensively 
explored in the literature about evaluation in information fusion 
systems (IFS). In this paper, we provide a discussion on confidence 
as it relates to the evaluation of IFS, and compare it with the existing 
concepts in the URREF ontology. Our goal is twofold, since we 
address both the distinctions between confidence and self-confidence, 
as well as the implications of these differences when evaluating the 
impact of uncertainty to the decision-making processes supported byt 
the IFS. We illustrate the discussion with an example of decision 
making that involves signal detection theory, confusion matrix fusion, 
subjective logic, and proportional conflict redistribution. We argue 
that uncertainty can be minimized through confidence (information 
evidence) and self-confidence (source agent) processing, The results 
here seek to enrich the ongoing discussion at the ISIF’s Evaluation of 
Techniques for Uncertainty Representation Working Group 
(ETURWG) on self-confidence and trust in information fusion 
systems design. 

Keywords: Self-Confidence, Confidence, Trust, Level 5 Fusion, High- 
Level Information Fusion , PCR5/6, Subjective Logic 

I. Introduction 

Information fusion aims to achieve uncertainty reduction 
through combining information from multiple complementary 
sources. The International Society of Information Fusion 
(ISIF) Evaluation of Techniques of Uncertainty Reasoning 
Working Group (ETURWG) was chartered to address the 
problem of evaluating fusion systems’ approaches to 
representing and reasoning with uncertainty. The working 
group developed the Uncertainty Representation and 
Reasoning Evaluation Framework (URREF) [1]. Discussions 
during 2013 explored the notions of credibility and reliability 
[2]. One recent issue is the difference between confidence and 
self-confidence as related to the data, source, and processing. 
While agreement is not complete among the ETURWG, this 
paper seeks to provide one possible approach to relate the 
mathematical, semantic, and theoretical challenges of 
confidence analysis. 

The key position of the paper is to analyze the practical 
differences in evaluating the two concepts. More specifically, 
self-confidence is mostly relevant to HUMINT, which makes 
its evaluation a primarily subjective; whereas confidence can 



be easily traced to machine data analysis, allowing for the use 
of objective metrics in its evaluation. That is, a machine can 
process large amounts of data to represent the state of the 
world, and the evaluation of how well uncertainty is captured 
in these processes can be traced to various objective metrics. 
In contrast, for a human to assess its own confidence on the 
credibility of his “data collection process” (i.e. self- 
confidence), he or she has to make a judgment on limited 
choices. Objective assessment is determined from the 
credibility of the reports, processing, and decisions. Typical 
approaches include artificial intelligence (AI) methods (e.g., 
Neural Networks), pattern recognition (e.g., Bayesian, 
wavelets), and automatic target exploitation (i.e., over sensor, 
target, and environment operating conditions [3]). Subjective 
analysis is a report quality opinion that factors in analysis 
(e.g., completeness; accuracy, and veracity), knowledge (e.g., 
representation, uncertainty, and reasoning), and judgment 
(e.g., intuition, experience, decision making) [4]. In terms of 
IFS support for decision-making, numerous methods have 
been explored, mainly from Bayesian reasoning, Dempster- 
Shafer Theory [5], Subjective Logic [6], DSmT [7], fuzzy 
logic and possibility theory; although it also includes research 
on approximating belief functions to subjective probability 
measures (BetP [8], DSmP [9]). 

Figure 1 provides a framework for our discussion. The world 
contains some truth T, of which data is provided from different 
sources (A, B). Source A analysis goes to a machine agent for 
information fusion processing while source B goes to a human 
agent. Either the machine or the human can generate beliefs 
about the state of the world (either using qualitative or 
quantitative semantics). The combination of A and B is a 
subject of Level 5 Fusion (user refinement) [10, 11]. 




Figure 1 - Information Fusion System assessment of confidence (machine) 
and self-confidence (sensor or human agent). 
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On one hand, confidence is typically related to machine 
processing such as signal detection theory where self- 
confidence is associated with sensors (and humans) assessing 
their own capability. On the other hand, the manipulation of 
the data requires understanding of the source, and self- 
confidence is applicable for the cases in which the user can 
provide a self-assessment on how confident he is on its data. It 
is important to emphasize that are not dealing (at least not 
directly) with information veracity: even if the sensor (e.g., a 
human reporting on an event or providing assessment on a 
situation) considers the information as possible, and he trusts 
it, it could be false at the end (e.g., even in summer time we 
can have a cloudy day). That is, self-confidence assesses how 
much the author trust the information, but not necessarily that 
this information is false or true [1]. For this paper, we take the 
URREF ontology definition of self-confidence as implied in 
Figure 1. The rationale for this choice is that self-confidence 
and uncertainty are typically associated with humans whereas 
confidence has been typically used in signal detection. Fusion 
of beliefs ultimately relates to states of the world with a 
reported confidence that can be compared to a truth state. 
Debating on the overlaps in terminology would be welcomed 
to clarify these positions for the ETURWG and the 
information community as a whole. 

From Figure 1, we note the importance of confidence as 
related from a decision to the estimated states. Self-confidence 
is within the human agent assessing their understanding (e.g. 
experience) that can also be combined with the computer 
agent. The issue at hand for a user is whether or not the 
machine analysis (or their own) state decision represents 
reality. The notion of reality comes from the fact that 
currently, there are many technical products that perceive the 
world for the user (e.g., video) from which the user must map 
a mental model to a physical model of what is being 
represented. Some cases are easy such as video of cars moving 
on a road [12]; however, others are complex such as cyber 
networks [13]. The example used through the rest of the paper 
requires High-Level Information Fusion (HLIF) of target 
detection from a machine and human [14, 15]. 

In designing computer-aided detection machines, it is 
desirable to provide intelligence amplification (IA) [16] where 
Qualia motivates subjective analysis as a relation between the 
human consciousness/self-awareness to external stimuli. 
Qualia is the internal perception of the subjective aspect of the 
human’s perception of the stimuli. Knowing oneself can then 
be utilized to understand/evaluate the use of meaningful and 
relevant data in decision-making. The more that a sensor 
understands its Qualia [17], the better it will be in providing an 
assessment of its self-confidence in a report or on a decision. 
Qualia then encompasses an important component to 
uncertainty reasoning associated with subjective beliefs, trust, 
and self-confidence in decision making as a sense of intuition. 
Not surprisingly, these are natural discussion topics in Level 5 
fusion (‘user refinement’), which includes operator-machine 
collaboration [18], situation awareness/assessment displays 
[19], and trust [20]. In order to explore self-confidence on 
these issues, we need to look at the psychology literature on 
trust as it relates to self-confidence. 



From data available on the web (e.g., twitter, documents), 
intelligent users need the capability to rapidly monitor and 
analyze event information over massive amounts of 
unstructured textual data [21]. Text from human sources is 
subjected to opinions, beliefs, and misperceptions, generating 
various forms of self-assumed self-confidence. In contrast, 
computer sensed data can be stochastic or deterministic, from 
which we have to coordinate the agent information. For 
example, with Gaussian observations generates stochastic 
probability analyses (e.g., Kalman Filter). However, structural 
information in the sensor models and sensitivities for a given 
state condition (which come from a deterministic ontology) 
could be used to improve the estimate [22]. This combination 
of both stochastic and deterministic decisions with uncertainty 
elements is usual in modeling and system deployment, and 
understanding its key aspects is a fertile area for producing 
better decision support from IFS. 

A related example from the analysis of uncertainty is evidence 
assessment from opinion makers. Dempster- Shafer theory has 
been used in connection with Bayesian analysis for 
decision making [5]. 

Likewise, Josang [23] demonstrated how subjective 
analysis within Dempster-Shafer theory could be used to 
determine the weight of opinions. Ontologies such as the 
one used in the URREF must be able to account for the 
uncertainty of data and to model it qualitatively, semantically, 
and quantitatively [24]. Metrics such as quality of service 
(QoS) and quality or information (IQ) are example of tools 
that can support and enhance a modeling capability between 
ontologies and uncertainty analysis [25]. The rest of this 
paper includes Sect. II as an overview of self-confidence. 
Sect. Ill discusses the mathematical analysis. Sect. IV 
highlights subjective logic for opinion making. Sect. V is an 
example and Sect. VI provides conclusions. 

II. URREF notions of Self-Confidence 

The ETURWG has explored many topics as related to a 
systems analysis of information fusion, which includes 
characteristics of uncertainty with many unknowns [26, 27]. In 
this paper, we categorize the characteristics of uncertainty into 
four areas, shown in Table 1. Assuming that the flow of 
information first goes from an agent to evidence beliefs, and 
subsequently to fusion with knowledge representation, then 
these areas help understand the terminology. Note that the 
defined information fusion quality of service (QoS) 
parameters are in blue {timeliness, accuracy, throughput, and 
confidence}. These could also be measures of performance 
[28]. For measures of effectiveness [25], one needs to 
understand system robustness (e.g., consistency, completeness, 
correctness, integrity). Here we focus on the red terms as 
related to self-confidence and confidence. 

Knowledge representation in IFS [29, 30] includes applying 
decision-making semantics to support the structuring of 
extracted information. One example is the use of well defined 
concepts (e.g. confirmed, probable, possible, doubtful, and 
improbable) to support information extraction with natural 
language processing (NLP) algorithms. As related to 
confidence and self-confidence, there is the notion of integrity. 



216 




Advances and Applications of DSmT for Information Fusion. Collected Works. Volume 4 



Table 1: Characteristics of Uncertainty 3 



Agent 


Evidence 


Algorithm 


Representation 


Source 


Information 


Fusion 


Knowledge Reasoning 






Scalability 


Knowledge Handling 


Objectivity 


Relevance 


Computational Cost 


Simplicity 


Observational Sensitivity 


Conclusiveness 


Adaptability 


Expressiveness 


Veracity (truthfulness) 


Veracity (truth) 


Traceability (pedigree) 


Polarity 


Secure 


Ambiguity 


Stability 


Modality 


Resilient 






Genericity 


Trust 


Precision 


Throughput 


Tense 




Accuracy 


Timeliness 




Reliability 

Self-Confidence 


Credibility 


Correctness 


Completeness 


Confidence 


Consistency 


Integrity 



This table is presented to the ETURWG in this paper to support ongoing discussions on the categorization of types of uncertainty 



Integrity for human agents is associated with their subjective 
accountability and consistency in making judgments. Integrity 
for a machine could be objective in the faithful representation 
and validity on the data [31]. 

Algorithm performance focus on the information fusion 
method. URREF criteria for evaluating it relates to how the 
uncertainty model performs operations with information. An 
example of related metrics is to assess uncertainty reduction 
by weighting good data over bad given conflicting data. 

Evidence : From [2], we explored the weight of evidence 
(WOE) as a function of reliability, credibility, relevance, and 
completeness. In URREF, WOE assesses how well an 
uncertainty representation technique captures the impact of an 
input affecting the processing and output of the IFS. 

Source : Self-confidence, while yet to have a clear definition in 
the engineering literature, is typically associated with trust. 

A. Trust 

Closely associated with subjective analysis is trust [32]. Trust 
includes many attributes for man-machine systems such as 
dependability (machine), competence (user), and application 
[33]. Trust is then related to machine processing (confidence) 
and human assessment (self-confidence). Trust in automation 
is a key attribute associated with machine-driven solutions. 
Human trust in automation determines a user’s reliance on 
automation. In [32], they explored self-confidence defined as 
the user anticipatory (or post) performance with machines 
which impacts with trust in policy application. 

Measuring trust as related to uncertainty is an open topic [34]. 
As a focus of discussion, we have a machine agent and a 
human agent of which a measure of trust comes from the 
uncertainty associated between the man-machine interactions. 
Reliability trust could be between human agents of which 
subjective probability is useful [35]. Decision trust could be 
between human agents or between a human and a machine and 
takes into account the risk associated with situation-dependent 
attitudes, attention, and workload of a human agent. The 
distinction between reliability and decision trust is important 
as related to self-confidence and confidence. This can be seen 
in Table 2, which depicts the main aspects for each of the six 
potential interactions between sensors.. 



Table 2: Trust Aspects in Sensor Interactions 





Human 


Others 


Machine 


Human 


Self-confidence 


Reliability 


Trustworthy 


Machine 


Trust 


Credibility 


Confidence 



• Human : Individuals must provide introspection on their 

own analysis and interaction with a machine. Here we 
distinguish between self-confidence and trust. In this 
case, human agents must have self-confidence in 
themselves as well as trust in the machine. 

• Others’. With the explosion of the Internet, recent work has 

explored the uncertainty of human sensing, such as 
Twitter reports in social networks, showing humans as 
less calibrated and reliable in their sensing. Wang et al. 
[36, 37] developed an estimation approach for truth 
discovery in this domain. Another recent example 
explored the decision-making trust between humans 
interfacing through a machine. The user interface was 
shown to have a strong impact on trust, cooperation, and 
situation awareness [38]. As an interesting result, 
credibility resulted as the computer interaction afforded 
complete and incomplete information towards 
understanding both the machine and the user analysis. 

• Machine : A large body of literature is devoted to network 

trust. Examples include the hardware, cyber networks 
[39], protocols and policies. Given the large amount of 
cyber attacks written by hackers, it comes down to a 
trustworthy network of confidentiality, integrity, and 
availability. For machine-machine processing without 
user-created malware, network engineering analysis is 
mostly one of confidence. Machine trust is also 
important to enterprise systems [40]. 

Since we seek to understand self-confidence as a URREF 
criterion, explorations included human processing and the 
human as a data source as shown in Figure 3. 




Figure 2 - Methods of Trust. 
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In the figure, multiple forms of trust are shown and related to 
the processing steps. Starting from the real world, data is 
placed on the network from which a machine (or sensor) 
processes the data. With context, a detection assessment is 
made for such domains as image, text, or cyber processing. 
That detection is the fused with the context information. For 
example, a detection of an object in an image is fused with 
contextual road information. The detection confidence is 
assessed and made available to the user with the context. The 
green line is the human-machine trust boundary as the human 
can look at the machine results or process the data themselves 
for given a level of confidence and render a decision. The 
dotted line then is a human assessment of whether or not the 
information presented represents reality and could be trusted. 

Note, if the human is the only sensor source, then he/she is 
looking at data and making a decision. Their self-confidence 
could be based on the machine results from which they factor 
in many types of trust. For example, context, as related to the 
real world (see Figure 1), provides a validation of the machine 
(network to algorithm trust as a measure of confidence), while 
at the same time understanding the situation to determine if the 
information fusion analysis is providing meaningful and useful 
information towards the application of interest. Together a 
trusted decision is rendered based on the many factors. 

Included in Figure 3 are many forms of trust in the analysis all 
of which can lead to confidence in the decision: 



Trust 


Processing 


Example 


Network 


Data put on a 
network 


Assessment of data timeliness and 
lost packets 


Machine 


Sensor 

transformation 


Calibration of cameras for image 
content 


Software 


Information 

management 


Getting the correct data from a 
data base (e.g., a priori data) 


Algorithm 


Fusion method 


Target tracking and classification 
results 


Modeling 


State models 


Kinematic and target recognition 
models (e.g., training data) 


Application 


Situation of interest 


Analysis over the correct area 
(e.g. target moving on a road) 


User 


Situation awareness 


Use of cultural and behavior (e.g. 
assume big cars move on roads). 



The self-confidence of the user analysis includes working with 
data, networks, and machines. The URREF ontology must 
account for trust over human-machine decisions for 
confidence analysis. To further explore how URREF is 
aligned with trust, we must look at self-confidence. 

B. Self-Confidence 

Statistically speaking, the machine decision-making accuracy 
is based on the data available, the model chosen, and the 
estimation uncertainty associated with the measured data. 
Given the above analysis, we could start to derive self- 
confidence for machine fusion operations based on the 
literature in human self-confidence. 

Self-confidence is the socio-psychological concept related to 
self-assuredness in one's personal judgment and ability. As an 
example, researchers are often called to review papers and 
after their review asked to give a quality rating of their own 
review based on their understanding of the subject, expertise, 



and experience. In another example, a person might be asked 
to identify an object in an image with a certain rating 
{unlikely, possible, probable, confirm} from which then they 
could determine self-confidence based on their answer. Thus, 
there is a need to assess “self-confidence” in relation to 
“confidence”, which is linked to uncertainty measures of trust. 

C. Accuracy and Precision 

Self confidence is strongly related to both precision and 
accuracy. A source can be self confident in both the precision 
of its generated data (consistency or variability in its reports - 
such as reported variance) as well as the accuracy of its reports 
(the reported bias or the reported distance of the mean value of 
the generated data form true value). In other words, to make 
sense of the term the self-confidence of a source, the data 
encapsulate a combination of precision and accuracy. A 
distinction is made between precision and accuracy reported 
by the machine (such as the estimated mean and variance at 
the output of the Kalman filter) and the actual precision and 
accuracy of the data emanating from the source. The URREF 
ontology categorizes accuracy, precision, and self-confidence 
as types of criteria to evaluate data [1]. 

Statistical methods of uncertainty analysis from measurement 
systems include accuracy and precision, shown in Figure 4. 
The use of distance metrics (accuracy) and precision metrics 
(standard deviations) help to analyze whether the 
measurement is calibrated and repeatable. We would desire 
the same analysis for human semantic analysis with precise 
meanings, consistent understanding, and accurate terminology. 



Value 




Figure 3 - Uncertainty as a function of accuracy and precision. 



Human Confidence- Accuracy: Traditionally known as the 
confidence-accuracy (CA) relationship, the assumption is that 
as one’s confidence increases so does their level of accuracy 
which is affected by memory, consistency, and ability [41]. 
Issues include absolute versus relative assessment, feedback, 
and performance. 

The confidence-accuracy relationship was shown to be a by- 
product of the consistency-correctness relationship: It is 
positive because the answers that are consistently chosen are 
generally correct, but negative when the wrong answers tend 
to be favored. The overconfidence bias stems from the 
reliability-validity discrepancy: Confidence monitors 

reliability (or self-consistency), but its accuracy is evaluated in 
calibration studies against correctness. Also, the response 
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speed is a frugal cue for self-consistency and depends on the 
validity of self-consistency in predicting performance [42]. 

Koriat [42] explains that Sensing tasks are dominated by 
Thurstonian uncertainty (local rank ordering with stochastic 
noise) within an individual and exhibit an under-confidence 
bias. However, general knowledge tasks are dominated by 
Brunswikian Uncertainty (global probabilistic model from 
limited sample sets to infer general knowledge [43]) that 
supports inter-person ecological relations. 

Consistency is then the repeatability of the information, which 
should imply no conflicts in decision-making. We can use the 
proportional conflict redistribution (PCR6) to get a measure of 
repeated consistency such that favored wrong answers are 
corrected in confidence analysis [44]. PCR6 is more general 
and efficient than PCR5 when combining more than two 
sources altogether. Moreover, PCR6 has been proved 
compatible with frequency probabilities when working with 
binary BBA's, whereas PCR5 and DS are not compatible with 
frequency probabilities [44]. 

Self-confidence could be measured with a Receiver Operating 
Characteristic (ROC) curve as once a decision can be made, 
we can then assess its impact on confidence. A low self- 
confidence would lead to chance, and a high self-confidence 
would remain to the left on the ROC. 



III. Self-Confidence 

Signal detection theory provides a measure of confidence in 
decision making that by assuming a limited hypothesis set is 
actually a measure of self-confidence. One classic example is 
Wald’s Sequential Probability Ratio Test (SPRT) [45]. 
Assuming evidence is sampled at discrete time intervals, then 
the human or cognitive agent compares the conditional 
probabilities x(t + At) for two hypothesis H j (j = 1, 2). Using 
then SPRT, then 



y{f) = h[x(i)\ = LN 



r /i Mon 
h WO] J 



( 1 ) 



It is noted that a choice in time is not just the product of the 
current analysis, but the accumulated evidence. For example, 
in Figure 5, we see that the signal is moving between semantic 
boundaries from doubtful to probable, with an associated 
measure label of possible. Given the history, then the decision 
maker could be self-confident in the current measurement 
given their perception of the entire processing of machine 
decision making measures for each time. 

A Piercian hypothesis [48] implies confidence is a 
multiplicative function of the quantity of the information 
needed to make a decision (#, or the distance traveled by the 
diffusion process) and the quality of the information ( S ; or the 
rate of evidence accumulation in the diffusion process) 
accumulated in Dynamic Signal Detection [48]. Without bias, 
the authors of [48] show that: 



conf (self) = (3 • 0 LN 



" PjRAjSA ~ 
. P(R B |S A ) _ 




(4) 



where p is a scaling parameter. A decision, 0, is related to a 
response ( R ) of detection to a stimuli (S). Given the ability to 
model self-confidence as a measure of precision, we extend 
the methodology using subjective-logic and DSmT [44] for 
robust decision making. 




Figure 4 - Evidence Accumulation for Decision Confidence. 



if y(t) > 0, then evidence supports H and if y(t) < 0, then H 2 
is more likely. As time accumulates for decision making, there 
is an aggregation of the log likelihood ratios: 



Lit + At) = L{t) + LN 



fi \x(t + At)] 

- f 2 [x(t + AO] _ 



( 2 ) 



where, for a stochastic system L(t) ~ Eq (2) can 

be written in Bayesian log odds: 



LN 



W ilW 
L p(H 2 | D) J 



Z LN 



\ h mi 1 
f 2 WO] J 



+ LN 



\eS2SL\ 
p(h 2 ) J 



(3) 



One then collects information to make a decision such that -0 2 
< L{t) < 0\. The chosen threshold is then a measure of a 
decision, which can be conservative or aggressive for the case 
of a human agent [46]. Figure 5 shows the case in which 
evidence is accumulated and a decision is made with 
associated standard boundaries for semantic decision making. 
Also in Figure 5 we related decision boundaries for semantic 
confidence classification [47]. 



IV. Subjective Opinons 

Subjective opinions [49] are special cases of belief functions 
as they correspond to bba defined on 2D frames of type 6 = 
{ A , —A} assuming Shafer’s model or DSmT. Subject opinions 
lend themselves to simple mathematical expressions of fusion 
models. We therefore use the opinion representation for 
describing the various fusion models, but the expressions can 
easily be mapped to traditional belief functions. 

A subjective opinion expresses belief about statements in a 
frame. Let X be a frame of cardinality k. An opinion 
distributes belief mass over the reduced powerset 7?(A) of 
cardinality k. The reduced powerset 7?(A) is defined as: 

R (X) = P(X) \ {V 0} , (5) 

where P{X) = 2 X denotes the powerset ofX. All proper subsets 
of X are elements of RQO, but the frame {X} and empty set 
{0} are not elements of R (A). 
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Let b x be a belief vector over the elements of R(X), u x be the 

complementary uncertainty mass, and ~a be a base rate vector 
over X. Whenever relevant, a superscript such as A denotes the 
opinion owner. Then a subjective opinion co^is the composite 
function expressed as: 

= ( b x , u x, a x) • (6) 

The attribute A is thus the belief source, and X is the target 
frame. The belief, uncertainty and base rate parameters satisfy 
the following additivity constraints. 

• Belief additivity: 



alternative interpretation of subjective opinions in terms of 
traditional statistics [6]. 

Cumulative Fusion: 

The cumulative fusion rule is equivalent to a posteriori 
updating of Dirichlet distributions. Its derivation is based on 
the bijective mapping between the belief and evidence 
notations described in [6]. 

The symbol “0” denotes the cumulative fusion of two 
observers A and B into a single imaginary observer AOB. 

Let ch 4 and CD 5 be opinions respectively held by agents A and B 
over the same frame X of cardinality k with reduced powerset 
R (X) of cardinality k. Let ch 4 05 be the opinion where: 



u x + Z b x (x 0 = 1, where x E R(X) (7) 

xi e R(X) 

• Base rate additivity: 
k 

X a x (* 0 = L where x EX (8) 

i = 1 

The belief vector Id x has k = (2* - 2) parameters, whereas the 

base rate vector ~a x only has k parameters. The uncertainty 
parameter u x is a simple scalar. A general opinion thus 
contains (2 k +k-\) parameters. However, given that Eq.(7) and 
Eq.(8) remove one degree of freedom each, opinions over a 
frame of cardinality k only have (2 k +k-?>) degrees of freedom. 
The probability projection of hyper opinions is the vector 

denoted as ~E? X • 

tx= x 7 X (x,- 1 xj) (Xj) + ~a x (X,) U x , V Xi e R(X) (9) 

xj e R(X) 

7 x (x,nx,) 

where a x (x z - 1 xj) = , V x u Xj cl. (10) 

a x (Xj) 

denotes relative base rate, i.e. the base rate of subset x z relative 
to the base rate of (partially) overlapping subset Xj. 

General opinions are also called hyper opinions. A 
multinomial opinion is when belief mass only applies to 
singleton statements in the frame. A binomial opinion is when 
the frame is binary. A dogmatic opinion is an opinion without 
uncertainty, i.e. where u = 0. A vacuous opinion is an opinion 
that only contains uncertainty, i.e. where u = 1 . Likewise, we 
can make the case that confidence in the opinion is biased by a 
subjective opinion of the source self-confidence. Thus, self- 
confidence is SCu = 1 owing to rank-order decision-making on 
a subset of the world, and the lack of self-confidence is SCu = 
0; where: 

SC„ («4) = ■fl X [1 -«x] (11) 

and co^ ^ — SCu (^U) • b X ( 1 2) 

Equivalent probabilistic representations of opinions, e.g. as a 
Beta pdf (probability density function) in case of binomial 
opinions, as a Dirichlet pdf in case of multinomial opinions, or 
as a hyper Dirichlet pdf in case of hyper opinions offer an 



CASE I: For u A ^ 0 v u B ^ 0 (with Confidence) 




_ b A (Xj) u B + b B (Xj) u A 
u A + u B — u A u B 

u A u B 

u A + u B - u A u B 



(13) 



CASE II: For u A = 0 v u B 0 (without Confidence) 



b** B (xd =y A b A (x l ) + y*bfix l ) 



A lA, 



tB( 



AOB 



= 0 



(14) 



where: i 



1 A ~ A 



Lil1 ^ 



o; o u A + u B 



y B - a 



Lm \ 



u ' —> o; o u A + u B 



Note: the case without confidence averages the results from 
self-confidence reports which weights effectively both the 
same. Confidence allows the user to weight the self- 
confidence of the reports based on the Brunswikian 
uncertainty about the world knowledge. 

Then ch 4 0 B is the cumulatively fused opinion of ctf 4 and CD 5 , 
representing the combination of independent opinions of A and 
B. By using the symbol ‘©’to designate this belief operator, 
cumulative fusion is expressed as: 

Cumulative Belief Fusion: co x A 0B = co x A © co x 5 (15) 



The cumulative fusion operator is commutative, associative 
and non-idempotent. In Eq.(15), the associativity depends on 
the preservation of relative weights of intermediate results 
through the weight variable y, in which case the cumulative 
rule is equivalent to the weighted average of probabilities. 



V. Results 

Assume we have two agent opinion makers (D A and CD 5 , who 
each make a decision for network security [50]. Let co A be a 
machine Algorithm and let CD 5 come from a human i?eing. 
After reporting their opinion, CD 5 is asked for their self- 

confidence. The result modifies their belief l? x , such that the 
cumulative belief fusion product is a weighted function of 
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their self-confidence (source) over their confidence (data). 
Figure 6 provides a perspective of the analysis. 




Figure 5 - Analysis With and Without Self Confidence. 

For many situations, a machine can process large amounts of 
data, while a human agent can only comprehend a subset of 
the data. Thus, a machine processes the data as outputs to a 
user. The interaction with the user is continually updated and 
a decision from the user is required. For situations in which 
the user has more time (forensics), then his/her self-confidence 
in the data would be high. For quick decisions, an observe- 
orient-decide-act (OODA) decision might be required [51] 
which reduces self-confidence. We seek methods of the latter 
as uncertainty is higher in rapid decision making which is a 
subset of problems in the Dynamic Data-Driven Application 
Systems (DDDAS) paradigm [52, 53]. 

For the analysis, we have two opinion makers (machine and 
man). Using signal detection theory, their individual measures 
of analysis provide a likelihood function. We then fuse the 
results with confusion matrix fusion [54] as a method of 
combination using Bayesian, Dempster- Shafer, or DSmT 
results [55]. We utilize two cases in which there is a high and 
low-confident observer (Case 1) and then the situation in 
which both have comparable analysis (Case 2). With two 
highly self-confident observers (Case 2), the results are similar 
to one of the observers which could be used for opinion 
validity. However, the user could be looking at the results and 
further analyzing context to provide a more appropriate 
analysis of their decision (e.g., based on culture, data 
completeness, etc). Using subjective logic the human being 
could modify their opinion, co 5 sc , which results in a larger 
value (e.g., know something) or lower value (e.g., recognize 
limitation of analysis). 

We assume that if the user provides no assessment of self- 
confidence, we provide equal weight to the results (average 
fusion). On the other hand, if a machine provides a measure 
of confidence, it could be derived from the dynamic-data, 
which we don’t simulate here. 

Example {High self-confidence with low self-confidence ) 
Assume that we have a highly self-confident opinion maker, 
( 0 A , that includes many sources and reliable analysis. On the 
other hand, we have a low-confident opinion maker, CD 5 , who 
is making a decision. When making their decision, CD 5 is 
guessing or almost chance, assuming that context provides 
pragmatic understanding of the world events. 

In Figure 7, there are two opinion makers, the red curve of a 
human agent suggesting that the result is “improbable,” while 
the more self-confident is in blue reporting “probable”. The 
fused result, shown in green, using self-confidence better 
reflects the true state; versus the average fusion of the opinion 
makers shown in magenta. The key issue is that self- 
confidence can help weight evidence. 



Estimation of Credibility Fused - DS 




Figure 6 - DS With (Fused) and With-out (Ave) Self-Confidence. 

Exploring DSmT [44], using the proportional conflict 
redistribution rule (PCR6 1 ), we also see in Figure 8 an 
improvement in the belief confidence when self-confidence is 
accounted for. 



Estimation of Credibility Fused - PCR5 




Figure 7 - PCR6 With (Fused) and With-out (Ave) Self-Confidence. 



VI. Conclusions 

In this paper, we assessed self-confidence as a criterion in the 
URREF. Self-confidence is typically associated with a source 
and relates a subjective quality on the rendering of their beliefs 
over data. For stochastic observations, we use the SPRT in a 
self-confidence analysis. However, to get the case of partial 
information, we use subjective logic for decision-makers. We 
demonstrated that the PCR6 is superior to DS for decision for 
a scenario in which a high self-confident observer opinion is 
fused with a low self-confident observer. Ultimately it is the 
user trust in the data they have available and opinions towards 
self-confidence; whereas a machine only reports confidence. 

Further directions include using the analysis with real 
operators doing intelligence analysis over data and associating 
semantic boundaries to their subjective decision-making. 



1 In the scenario, we used sequential fusion of two sources and because of this, 
PCR5=PCR6, i.e. when combining 2 sources only PCR5 coincides with 
PCR6. 
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